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Abstract 

Queueing networks are gaining attraction for the performance analysis of parallel computer sys- 
tems. A Jackson network is a set of interconnected servers, where the completion of a job at 
server i may result in the creation of a new job for server j. We propose to extend Jackson 
networks by "branching" and by "control" features. Both extensions are new and substantially 
expand the modelling power of Jackson networks. On the other hand, the extensions raise com- 
putational questions, particularly concerning the stability of the networks, i.e, the ergodicity of 
the underlying Markov chain. We show for our extended model that it is decidable in polyno- 
mial time if there exists a controller that achieves stability. Moreover, if such a controller exists, 
one can efficiently compute a static randomized controller which stabilizes the network in a very 
strong sense; in particular, all moments of the queue sizes are finite. 

1998 ACM Subject Classification G.3 Probability and Statistics 

Keywords and phrases continuous-time Markov decision processes, infinite-state systems, per- 
formance analysis 

|l Introduction 

Queueing theory plays a central role in the performance analysis of computer systems. In 
particular, queueing networks are gaining attraction as models of parallel systems. A queueing 
network is a set of processing units (called servers) , each of which performs tasks (called 
jobs) of a certain type. Each server has its own queue of jobs waiting to be processed. The 
successful completion of a job may trigger one (or more) new jobs (of possibly different type) 
that need to be processed as well. In addition to this "internal" job creation, so-called open 
queueing networks allow for new jobs to arrive "externally", i.e., from outside. 

Queueing networks are a popular model for both hardware and software systems because 
of their simplicity and generality. On the hardware side, queueing networks can, e.g., be used 
for modeling multi-core processors, see e.g. [20] and the references in [9^. One advantage of 
queueing-based analyses is their scalability with growing parallelism; e.g., it is said in [5T]: 
"Cycle-accurate full-system performance simulators do not scale well beyond a few tens of 
processor cores at best. As such, analytical models based on the theory of queueing systems, 
are a logical choice for developing a basic understanding of the fundamental tradeoffs in 
future, large-scale multi-core systems." On the software side, queueing networks are used for 
modeling message passing. It is said in [29] : "Two natural classes of systems can be modeled 
using such a framework: asynchronous programs on a multi-core computer and distributed 
programs communicating on a network." Of course, the realm of queueing networks stretches 
far beyond computer science, see [SJ[S]. 
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The simplest queueing networks are so-called Jackson networks [16 : Given two servers 

Pij 

j, J e {1, . . . , n}, there is a "rule" of the form i ' — >■ j which specifies the probability pij that 
the completion of an i-job results in the creation of a j-job. There are also rules i e 
where Pio — 1 — Pij specifies the probability that no new job is created. Each server i has 
a rate with which an i-job is processed if there is one. In addition, there is a rate with 
which «-jobs arrive from outside the network. The processing times and the external arrivals 
are exponentially distributed, so that a Jackson network describes a continuous-time Markov 
chain ( CTMC). It was shown in Jackson's paper [TB] that if the rate of internal and 
external arrivals at server i is less than /i^ for all i, then the network is stable, i.e., the average 
queue length is finite and almost surely all queues are empty infinitely often. Moreover, 
Jackson networks allow a product form, i.e., the steady-state distribution of the queue lengths 
can be expressed as a product of functions ni{k), where 7ri(fc) is the steady-state probability 
that queue i has length k. 

► Example 1 (network processor). In [Ij, Jackson networks are used to model network 
processors, i.e., chips that are specifically targeted at networking applications — think of a 
router. We describe the model from [T] (sections 4.1 and 4.2, slightly adapted). Before 
packets are processed in the "master processor" M, they pass through the "data plane" D, 
from which a fraction q of packets needs to be processed first in the "control plane" C: 

D M dAc C a M M a £ 

An "arrival manager" A sends some packets (fraction c?o) directly to D, but others (frac- 
tions di,. . . ,dn with c?o -I- di -I- • • • -|- (in = 1) are sent to "slave processors" Si, . . . ,Sn to assist 
the master. Some packets (fraction h) still need the attention of the master after having been 
processed by a slave: 

A-^D A A 5, S,Ad S.'^e , zG{l,...,n}. 

Jackson networks and their extensions have been thoroughly studied, but they are 
restricted in their modelling capabilities, as (i) the completion of a job may trigger at 
most one job, and (ii) there is no nondeterminism that would allow to control the output 
probabilities of a server. Considering (i), it seems unnatural to assume that a distributed 
program communicating on a network produces at most one message at the end of its 
computation. Considering (ii), the "arrival manager" A in Example [l] may want to flexibly 
pass incoming packets to the master or one of the slaves, possibly depending on the current 
load. These restrictions have not been fully addressed, not even in isolation. In this paper we 
introduce controlled branching queueing networks, which are Jackson-like networks but allow 
for both nondeterminism ("controlled") and the creation of more than one job ("branching"). 

Both extensions directly raise computational issues. We show in Example [2] on page [5] 
that even purely stochastic branching networks do not allow a product form, which illustrates 
the mathematical challenge^ of this extension and poses the question for an effective criterion 
that allows to determine whether the network is stable, i.e., returns to the empty state 
infinitely often. Moreover, due to the nondeterminism, we now deal with continuous-time 
Markov decision processes (CTMDPs). Our main theorem (Theorem [s]) states that if there 
exists any scheduler resolving the nondeterminism in such a way that the controlled branching 



It is noted in [15] that "[...] virtually all of the models that have been successfully analyzed in classical 
queueing network theory are models having a so-called product form stationary distribution." 
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network is stable, then there exists a randomized static scheduler that achieves stability as 
well, where by "randomized static" we mean that the decisions may be randomized but not 
dependent on the current state (such as the load) of the system. Moreover, the existence of 
such a stabilizing scheduler and the scheduler itself can be determined in polynomial time, 
and, finally, the randomized static scheduler is stabilizing in a very strong sense, in particular, 
all moments of the queue sizes are finite. 

Related work. We use nondeterminism to describe systems whose behaviour is not 
completely specified. A system designer can then resolve the nondeterminism to achieve 
certain goals, in our case stability. Although nondeterminism is a very well established 
modelling feature of probabilistic systems (see e.g. the literature on automatic design 

of stabilizing controllers for queueing networks is sparse. Flow-controlled networks |28| 122] 
allow to control only the external arrival stream or the service rates (see also [2] and the 
references therein) . The authors of |19| 114] consider queueing networks with fewer servers 
than job types, so that the controller needs to assign servers to queues. As in [EJUl], we 
also use linear programming to design a controller, but our aim is different: we allow the 
controller to influence the production of the individual queues, and we study the complexity 
of designing stabilizing controllers and the nature of such controllers. There has been a 
substantial amount of work in the last years analyzing probabilistic systems with "branching 
features", most prominently on recursive Markov chains fT3[ I12j and probabilistic pushdown 
systems [TTl |S] . While these models allow for a probabilistic splitting of tasks by pushing 
new procedures on a stack, the produced tasks are processed in a strictly sequential manner, 
whereas the queues in a queueing network process jobs in parallel and in continuous time. 
Recently, probabilistic split-join systems were introduced [17j . which allow for branching but 
not for external arrivals, and assume unlimited parallelism. In [181 chapter 8] a queueing 
model with multiple classes of tasks and "feedback" is discussed, which is similar to our 
branching except that there is only one server, hence there is no parallelism. Algorithmic 
theory of queueing systems has also attracted some attention in the past. In particular, for 
closed (i.e., without external arrivals) queueing systems, [26j shows EXP-completeness of 
minimizing a weighted throughput of the queues. 

1 2 Preliminaries 

Numbers. We use Z,Q,M for the sets of integer, rational, real numbers, respectively, and 
N,Q>o,]R>o for their respective subsets of nonnegative numbers. 

Vectors and Matrices. Let n > 1. We use boldface letters for denoting vectors x — 
{xi, . . . ,Xn) € M". Vectors are row vectors per default, and we use superscript ^ for 
transpose, so that a;-^ denotes a column vector. If the dimension n is clear from the context, 
we write := (0, . . . , 0), 1 := (1, . . . , 1), and e(*' = (0, . . . , 0, 1, 0, . . . , 0) for the vector with 
the 1 at the ith component (1 < i < n). It is convenient to define e^^^ := 0. For two 
vectors x,y ^ M" we write x ^ y with ~ € {=, <, <, >, >} if the respective relation holds 
componentwise. For a vector x S M" we denote its 1-norm by |la;|l := Yll^i When 
X G N" is a vector of queue sizes, we refer to ||a;|| as the total queue size. For a matrix 
A e M"^", we write Ai for its ith row, i.e., Ai = (Aa, . . . , Ai„). 

CTMDP. A continuous-time Markov decision process ( CTMDP) consists of an at most 
countable set S of states, an initial state Si G 5', a set of actions E ^, and a transition rate 



Usually, each state has its own set of available actions. As this feature is not needed for queueing 
networks, we stick to the simpler version in which all actions are always available. 



4 



Stabilization of Branching Queueing Networks 



q{s, a, s') > for each pair of states s, s' e 5 and each action cr e S (here q{s, <t, s') = 
means that the transition from s to s' never occurs). We define a continuous-time Markov 
chain ( CTMC) to be a CTMDP whose set of actions S is a singleton (we usually do not 
write the only action explicitly, so the transition rates of a CTMC are denoted by q{s, s'), 
etc.). 

Intuitively, a run of a CTMDP starts in si and then evolves in so-called epochs. Assume 
that (after the previous epoch) the system is in a state s. The next epoch consists of the 
following phases: First, a scheduler chooses an action ct S S to be executed. Second, a waiting 
time for transition to each state s' € 5 is chosen according to the exponential distribution 
with the rate q{s, cr, s') (here we assume that if q{s, cr, s') = 0, then the waiting time is oo). 
The transitions compete in a way such that the one with the least waiting time is executed 
and the state of the CTMDP is chosen accordingly (the other transitions are subsequently 
discarded) . 

Formally, a run is an infinite sequence si, ci, ^i, S2, cr2, ^2, • • ■ & (5* x E x R>o)'^. We 
denote by Run the set of all runs. A scheduler is a function which assigns to every finite 
path si, (Ti, ii, S2, (72, ^2, ■ ■ • 5,1 e (5* X S X M>o)* x S" a probability distribution Q(w) on 
actions (i.e. Q{w) : E ^ [0, 1] satisfies X^o-gs 'd{w){cr) — 1). For technical reasons, we have 
to restrict ourselves to measurable schedulers (for details see e.g. pi]). 

We work with a measurable space of runs {Run,F) where is the smallest a-algebra 
generated by basic cylinders (i.e. sets of runs with common finite prefix) in a standard 
way. Every scheduler Q induces a unique probability measure Pre on T determined by the 
probabilities of the basic cylinders. For detailed definitions see Then each scheduler O 
induces a stochastic process {x{t) \ t € M>o) on the probability space (i?M7i, Pre) where 
x{t) is the current state of the run in time t, i.e., each x{t) is a random variable defined by 



A scheduler O is memoryless if for every path w — si, cti, ii, S2, 172, ^2, • • ■ Sn+i G (S* x E x 
]R>o)* X S* we have that Q{w) = 6(s„+i). 

Networks. Define R("'^) :— {r G N" | ri + • • • + r„ < K}. A production function for 
(n, K) is a function Prob : i? ^ Q n (0, 1] with R C Ri"--^) such that J^reR Prob{r) = 1. A 
controlled branching network with n queues and branching factor K consists of an arrival 
rate /Lto S Q H (0, 00), queue rates /i^ € Q n (0, 00) for i g {1, . . . , n}, an arrival production 
function Probg : i?o Q H (0, 1] for (n, K), and finite action sets Ei, . . . , E„ as follows. An 
action ai £ E^ assigns to queue i a production function Probi{ai) : Ri{ai) ^ Q H (0, 1] for 
{n,K). Define E := Ei x • • • x E„. If cr = (tri, . . . ,(7n) G S, we write Ri(a), Probi{a) and 
Ro{cf), Probo{<y) to mean Ri{cri), Probi{ai) and Rq, Probo- Observe that the rates fii do not 
depend on actions. This simplification is without loss of generality.'^ We assume a nonzero 
arrival stream, i.e., there is r G i?o with r ^ 0. We define the size of a controlled network 



by n + if + (ELo \^^^\) + \Ro\ + \Probo\ + Eti + where etc. 



means the description size assuming the rationals are represented as fractions of integers in 



^ To show that this assumption is w.l.o.g. one can employ the standard "uniformization" trick. More 
precisely, assume that the actions of a queue i have different rates. Define to be the maximum of 
all rates of Ei and compensate by "adding self-loops", i.e., make the actions of Ei generate a new job 
for queue i with a suitable probability. This effectively substitutes a transition with longer delay by 
possibly several transitions with delay fii. As static schedulers can be easily translated between the 
original and the transformed system, our results remain valid. 
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binary. A controlled branching network induces a CTMDP with state space N" (the queue 
sizes), initial state 0, action set E, and transition rates 

q{x, (J,y)^ Yl iJiiProbi{G){r) for x, y g N", cr e E. 

Interpreting this definition, there is a "race" between external arrivals (rate /io) ^ind the 
nonempty queues (rates /i^); if the external arrivals win, new jobs are added according 
to Probo{a)] if queue i wins, one i-job is removed and new jobs are added according 
to Probi{a). 

An purely stochastic branching network is a controlled branching network with S = {ct}, 
i.e., with a unique action for each queue. Hence, the induced CTMDP is a CTMC. In the 
purely stochastic case we write only Ri for Ri{a) etc. If Probi{r) — pin the purely stochastic 

case, we use in examples the notation i ^ r, where we often write r e R("'^) as a multiset 

p p p 

without curly brackets. For instance, if n = 2, we write 1^1,2 and 1^2,2 and 1 ^ e to 

mean Probi{r) = p with r — (1, 1) and r = (0, 2) and r = (0, 0), respectively. 

Fixing a controlled network Af and a scheduler Q for the CTMDP induced by Af, we 

obtain a stochastic process Mq = {x{t) \ t G M>o), where a;(0) = € N", which evolves 

according to the dynamics of M and the scheduler O. In the purely stochastic case we drop 

the subscript Q, and so we identify a network J\f with its induced stochastic process. 

► Example 2 (no product form). Consider the purely stochastic branching network with 

A- 1, 2 and 1 A- e and 2 A- e. If its stationary distribution tt (for a definition of stationary 
distribution see before Theorem |3| had product from, the queues would be "independent in 
steady-state", i.e., tt{x2 > 1 | a^i > 1) = tt{x2 > 1), where by x we mean x{t) in steady state. 
However, if fiQ is much smaller than fii — /12, then we have n(x2 > I \ Xi > I) > ■k{x2 > 1), 
intuitively because Xi > 1 probably means that there was an arrival recently, so that X2 > I 
is more likely than usual. More concretely, let /zq = 1 and /ii = /i2 = 3 and consider the 
2-state Markov chain obtained by assuming that each arrival leads to the state (1,1) and each 
completion of any job leads to the state (0,0). By computing the stationary distribution tt' of 
this 2-state Markov chain in the standard way, we obtain 7r'((0, 0)) = 6/7 and 7r'((l, 1)) — 1/7. 
Since this 2-state Markov chain "underapproximates" the CTMC induced by the network, 
we have 7t{xi > 1 A 2:2 > 1) > 1/7. On the other hand, by considering the two queues 
separately, the standard formula for the M/M/1 queue gives Tr{xi > 1) = tt{x2 > 1) = 1/3. 
Product form would imply tt{xi > 1 A 0:2 > 1) = t^{xi > 1)-tt{x2 > 1) = 1/9, contradicting 
the inequality above. 

f3 Results 

We focus on the stability of purely stochastic and controlled branching networks. Our notion 
of stability requires that the network is completely empty infinitely many times. Given a 
stochastic process {x(t) \ t € M>o), we say that the process is ergodic if the expected return 
time to is finite. More formally, define a random variable R by 

R := inf {t I x(t) ^0,3t' <t: x(t') ^ 0} . 

t>o 

Then the process is ergodic iff E [i?] < 00. In the controlled case, we say that a scheduler O 
for Af is ergodic for Af if A/e is ergodic. In the following we use stability and ergodicity 
interchangeably. A scheduler G is static if it always chooses the same fixed distribution on 
actions. Note that static schedulers are memoryless. If in a stochastic process {x{t) \ t E M>o) 
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the limit Tr{x) :— limt_j.oo Pr(a;(i) = a;) exists for all x e N" and X)a;eN" "■(^j) — 1, then 
TT : M>o — > [0, 1] is called the stationary distribution. 

► Theorem 3. Let M be a controlled branching network. It is decidable in polynomial time 
whether there exists an (arbitrary) ergodic scheduler for M . If it exists, one can compute, in 
polynomial time, a static randomized ergodic scheduler for J\f with stationary distribution tt 
such that there exists an exponential moment of the total number of waiting jobs, i.e., there 
is S > such that X)xeN" exp((5 ||a;||)7r(a;) exists. 

To prove Theorem |3] we generalize the concept of traffic equations (see e.g. [J) from the 
theory of Jackson networks. Intuitively, the traffic equations express the fact that the inflow 
of jobs to a given queue must be equal to the outflow. Remarkably, the traffic equations 
characterize the stability of the Jackson network. More precisely, a Jackson network is stable 
if and only if there is a solution of the traffic equations whose components are strictly smaller 
than the rates of the corresponding queues (we call such a solution deficient). 

We show how to extend the traffic equations so that they characterize the stability of 
controlled branching networks. For a smooth presentation, we start with purely stochastic 
branching networks and add control later on. Hence, the overall plan of the proof of Theorem|3] 
is as follows: Set up traffic equations for purely stochastic branching networks and show 
that if there is a deficient solution of these equations, then the network is stable. This 



result, presented in Section 3.1 (Proposition |4]) , is of independent interest and requires the 
construction of a suitable Lyapunov function. Then, in Section [3.2[ we generalize the traffic 
equations to controlled branching networks and show that any ergodic scheduler determines 



a deficient solution (Proposition 10 1. This solution naturally induces a static scheduler, 
which, when fixed, determines an purely stochastic network with deficiently solvable traffic 
equations. Propositions [4| and |10| imply Theorem [3] and provide some additional results. 



3.1 Purely stochastic branching networks 

Assume that Af is purely stochastic, i.e., there is a single action for each queue. In such a case 
the CTMDP induced by the network is in fact a CTMC. We associate the following quantities 
to a network, which will turn out to be crucial for its performance. Let /x :— (/ii, . . . , 
Let a (z ]R>Q be the vector with cti : — MoX^re-Ro ^^^^oif)"^!^ oli indicates the expected 
number of external arrivals at queue i per time unit. Note that a 0, as we assume a 
nonzero arrival stream. Let A E E"g" be the matrix with Aij :— J^reR Probi{r)rj; i.e., Aij 
indicates the expected production of j-jobs when queue i fires. W.l.o.g. we assume that all 
queues are "reachable", i.e., for all queues i there is j G N with {aA^)i ^ 0. We define a set 
of traffic equations 

n 

Xj = aj+'^X.-Aij , j e {1, . . . ,n}, (1) 
in matrix form: 



X = a + XA. (2) 

We prove the following proposition. 

► Proposition 4. Assume that A e M"q solves the traffic equations Q and satisfies X < fj,. 
Then the following conclusions hold: 

1. The process M is ergodic, i.e., the expected return time to is finite. 
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2. There exists a stationary distribution vr such that there exists an exponential moment of 

the total queue size, i.e., there is S > such that X^xeN" 6xp((5 ||a;||)7r(a;) exists. 
The key step to the proof of Proposition |4] is to construct a so-called Lyapunov function with 
respect to which the process Af exhibits a "negative drift". This is in fact a classical technique 
for showing the stability of queueing systems [53]; the difficulty lies in finding a suitable 
Lyapunov function. The "drift" of Af is given by the mean velocity vector A(a;) G M"q 
of Af, defined by A(a;) :— lim/j_j.o+ ^ [x{t + h) ~ x{t) \ x{t) = x] /h. The limit exists, is 
independent of t, and we have 

A{x)^a+ Ai»(-e«+^0- (3) 
The following lemma is implicitly proved in [10 , theorem 1]. 

► Lemma 5. Suppose that a function V : M"q — >■ M>o is two times continuously differentiable, 
V{x) = implies x — 0, and that there is 7 > such that we have 

A{x){V'{x))^ <--f forallx^O, 

where V'{x) denotes the gradient of V at x. Then the conclusions of Proposition^ holds. 

Following [TU] , we construct the Lyapunov function V in two stages: we first define a suitable 
piecewise linear function V : N" — >■ M>o; then V is smoothed to obtain V. For the definition 
of V we need the following lemma. 

► Lemma 6. The matrix series A* :— X^i^o converges ("exists") in M"q" and is equal to 

{i-Ar\ 

Define vectors . . , q^") € by setting a(*)^/ ||a^'^||, where a^*)'^ is the ii\\ 

column of A* . Observe that we have Iq'^*)-^ — 1 for all i. Define the function V : M"q ^> IR>o 
by V{x) := maxj ja;*/'*''-'"}. We will use the following property of V: 

► Lemma 7. IfO^xe M"(, and Xi = 0, then a;q(*)'^ < V{x). 

Lemma [7] is not obvious; in the appendix we use Farkas' lemma for the proof. The following 
lemma describes the crucial "negative drift" property of V: 

► Lemma 8. There is 7 > such that we have 

A{x){V'{x))^ <--f forallx^O 
and all subgradient vectors V'{x) of V at x. More precisely, one can choose 
7 min(^i - Xi)/ 

i 

where a^'^-^" is the ith column of A* . 

► Example 9. Consider the network with 

1/5 1/6 

04l 1^2,2 2^-^1,2 
" ^ 4/5 5/6 ' 

1 2-— >£ 

arrival rate /io — 7/30, and = (5/12,7/20), so that fiQ + fii + fi2 = 1- Let us write 
[0] := a = Mo(l,0), [1] := Aii(-e(i) + A,), [2] ^,2{-e(^^ + A2), [01] := [0] + [1], 
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[02] := [0] + [2], [012] := [0] + [1] + [2]. These vectors are shown in Figure [T] (a) . The 
mean velocity vector A{x) is one of the vectors [0], [01], [02], [012], depending on which 
components of x are nonzero. The vector field in Figure [l] (b) shows the corresponding 
vectors for several a; G N^. The connected line segments indicate points x with the same 
value of V{x) = max{a;q(-^^^, xq^^^"^} — max{|a;i + ^X2, ^Xi + |x2} (values 0.5, 1, 1.5, . . .). 
It can be seen from the figure that the drift is negative with respect to the gradient of V, if 
x^O. 

Proof of Lemma |5J Let x =/= 0. We need to show A{x)q^^^'^ < —7 for all i with xq^^^'^ — 
V{x). W.l.o.g. we assume that xq^^^'^ = V{x) and show only A.{x)q^^^'^ < —7. By Lemma]?] 
we have Xi ^ 0. It follows from the property (/ — A)A* — I and the definition of q^^-' that 
we have 



(_e(i)+Ai)q(i)^ = -l/ ad 



and 



(-e(*) + = for 2 < i < n. (4) 



Hence we have: 



aq 



(1)T 



Ml/ k' 



(1) 



,.(1) 



< -7 + aqd)^ - Ai/ 

= -7+(a + A(-/ + A))q(i)^ 
= -7 + Oq(^'^ = -7 



by (§ 

by Q and Xi ^ 
by the definition of 7 

by ^ 

by the traffic equation. 
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Using integration, one can obtain a two times continuously differentiable function V satisfying 
the conditions in Lemma [sj the function V is smoothed by defining V{x), for all x, as an 
"average" of the values V{y) where y belongs to a small ball around x; see the appendix 
of [in] for the formal details. This concludes the proof of Proposition |4j 

3.2 Controlled branching networks 

In this subsection we generalize the traffic equations ([2| to deal with an arbitrary controlled 
branching network M. To obtain a distribution on actions for a static randomized ergodic 
scheduler, we assign variables to actions instead of queues, i.e., for every action ^ of the 
network we introduce a variable capturing the rate of firing the action ^. Denote by S 
the set ljr=i ^i- Given ( &T, and j € {1, . . . ,n}, we denote by A(^j the average number of 
jobs added to the queue j when the action C fires, i.e., for C G S.^ we set 

We generalize (|2| to the traffic LP presented in Figure [2] where the variable S is intended to 
bound, for all j, the probability that queue j is busy. 



min S subject to 



i=i ceSi 



S > 



Af > 



j e {!,..., n} 
j e {!,..., n} 



■ Figure 2 The traffic LP. 



We prove the following 
► Proposition 10. 

1. If there exists an arbitrary ergodic scheduler for M , then the traffic LP can be solved with 
min S < I. 

2. If the traffic LP is solved with min (5 < 1, one can compute in polynomial time a 
static randomized ergodic scheduler for J\f. Moreover, denoting by pi the utiliza- 
tion limt_j.oo Pr(£Ci(i) ^ 0) of the queue i, the scheduler minimizes max^ pi among all 
memoryless ergodic schedulers. 

Hence one can decide in polynomial time whether an arbitrary ergodic scheduler exists; if 
yes, one can compute in polynomial time a static randomized ergodic scheduler. 

Let us first concentrate on part 1. Let O be an ergodic scheduler. Roughly speaking, we 
prove that a feasible solution of the traffic LP can be constructed using (limit) frequencies of 
firing individual actions in Mq. Formally, given a run w of A/e, t g M>o and ^ S S, we denote 
by 0^*(a;) the number of times the action ^ is fired up to time i on w. For memoryless Q we 
have the following result. 
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► Lemma 11. Assume that is a memoryless ergodic scheduler. For every ^ G S there is a 
constant such that for almost all runs uj of Afe the limit 

Of(c.) 

iim — ^ 

t— >oo t 



exists and is equal to O^. There is 5 < 1 such that {S,0^ | ^ G solves the traffic LP. 
Moreover, for every i G {1, . . . , n} the utilization pi of the queue i in A/e is equal to 



We prove Lemma [llj in Appendix A. 3 If there exists an arbitrary (i.e. possibly history- 
dependent) ergodic scheduler, then by Theorem 7.3.8 of [27] there exists also a memoryless 



(and deterministic) ergodic scheduler.^ This fact, combined with Lemma 11 implies part 1. 
of Proposition [TOj 

Now let us concentrate on part 2. of Proposition [T0| 

► Lemma 12. Any feasible solution {S,X^ | ^ G S) of the traffic LP with S < 1 induces 

static randomized ergodic scheduler whose utilization of any queue i is equal to 



y h 



Mi 

Proof. We construct a static randomized scheduler O which chooses an action ^ e for 
the queue i with probability 

Pi- ^^' T ' E^C>0. (5) 

Otherwise, if '^C ~ may control the queue i arbitrarily because no jobs ever 

come to the queue. We further assume (w.l.o.g.) that such queues have been removed from 
the network, i.e., that is defined using (jHj) for all ^ e S. Note that X^^es -^4 = 1 every 
i £ {1, . . . ,n}. 

Fixing the scheduler we obtain a purely stochastic branching network whose traffic 
equations are deficiently solvable. Formally, we define a new purely stochastic branching 
network Af' with n queues with the same arrival rate, the same arrival production function 
and the same queue rates as Af. Further, Af' has R'^ = U^es P-iiO ^"^^ the following 
production functions Proh,i associated to queues: 

Proh',{r) ^ ^ P^ ■ Probi{£){r) , r e R[ 

(Here we formally assume Probi{S^){r) = for r ^ i?^.) The traffic equations ([ij for TV' have 
the following form: 

n 

= "j+I^A, -A^ , je{l,...,n} (6) 

i=l 

with 

A[^ := E ^™^^(^) • = E ^? E P^^b^iOir) ■ r, = 

- E E ^-MO(r) • r,^ E • % • 



* To be formally correct, we apply Theorem 7.3.8 of J27' to the embedded discrete time MDP and obtain 
a scheduler which returns to the state in finitely many steps (on average). As there are only finitely 
many rates in our system, this means that also the expected return time to is finite. 
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Setting Xi :— X^^es ^S, every i e {1, . . . ,n}, we obtain XiA'^j — X)^es ^i^U- P^^ 
this equality into the first equation of the traffic LP, we see that (Ai, . . . , A„) solves ([6|. Also, 
\j < fij for all j G {1, . . . , n}. Proposition |4] then implies that the scheduler Q is ergodic. 

Finally, let us concentrate on the utilization. Note that the utilization of any queue i is 
the same in Af' as in Ae, so it suffices to concentrate on Af' . Observe that the matrix I — A' 
is invertible by Lemma |6] This means that (Ai, . . . , A„) is, in fact, the unique solution of ([6]). 
Then however, by Lemma the utilization pi of queue i in Af' (and thus also in A/e) is 



equal to 



To complete the proof of Proposition 10 we consider the problem of minimizing the maximal 



utilization max^ pi. Let be a static randomized ergodic scheduler induced by a solution 



of the traffic LP in the sense of Lemma 12 (here we consider a solution which minimizes 5). 
Observe that the scheduler minimizes niax^ pi among all schedulers induced by solutions 
of the traffic LP. However, by Lemmas 1 11 1 and 1 1 2 [ for every memoryless scheduler G there 
exists a static randomized scheduler induced by a solution of the traffic LP which has the 
same utilization of each queue as 0. Thus 0^ minimizes max^ pi among all memoryless 
ergodic schedulers. 



f4 Conclusions 

We have suggested and studied controlled branching networks, a queueing model which 
extends Jackson networks by nondeterministic and branching features as required to model 
parallel systems. Although much of the classical theory (such as product-form stationary 
distributions) no longer holds for controlled branching networks, we have shown that the 
traffic equations can be generalized. This enabled us to construct a suitable Lyapunov 
function which we have used to establish strong stability properties. We have shown for 
the controlled model that static randomized schedulers are sufficient to achieve those strong 
stability properties. Linear programming can be used to efficiently compute such a scheduler, 
which at the same time minimizes the maximal queue utilization. 

Future work should include the investigation of more performance measures, e.g., the 
long-time average queue size. Can non-static schedulers help to minimize it? 
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I A Proofs 

A.l Proof of Lemma [5] 



Lemma (n 



The matrix series A* 



T^Zo"^' converges ("exists") m K!^^" 



and is equal to 



Proof. By the trafRc equation X = a. + XA we have XA < X, with A > 0. By [3, Theo- 
rem 2.1.11] this implies that the spectral radius of A is at most 1 (where the spectral radius 
is the largest absolute value of the eigenvalues of A) . 

To show the statement of the lemma, it now suffices to show that I — A is invertible. 
Consider the (monotone) function / : M"q M"g with f{x) := a + xA. By the traffic 
equation, A is a fixed point of /. Assume for a contradiction that / — A is singular, i.e., there 
is y € M" with y ^ and yA = y. Then A + ry is a fixed point for all r e M. Choose r 



so that z 



ry e 



but Zi — Q for some i G {1, . . . , n}. By the monotonicity of /, 
all points u € {0, /(O), /(/(O)), . . .} satisfy <u < z = f{z). It follows that u, = for 
all u. This contradicts the fact that queue i is "reachable" (recall that "queue i is reachable" 
means there is j G N with {cxA^)i ^ 0). M 



A. 2 Proof of Proposition [4] 

In this section we complete the proof of 

Proposition |4| Assume that X G R>q solves the traffic equations ([2| and satisfies X < fi. 
Then the following conclusions hold: 

1. The process J\f is ergodic, i.e., the expected return time to is finite. 

2. There exists a stationary distribution vr such that there exists an exponential moment of 
the total queue size, i.e., there is S > such that X^xgN" 6xp((5 ||a;||)7r(a;) exists. 

Recall from LemmaKJ^that the matrix series A* :— X^i^o converges ("exists") in M"q" 
and equals (/ — A)~^ .(Jhserve that for any vector — A*w^ with w G M", we have 
v"^ = Av^ + wr'" . We prove Lemma [t] 

Lemma [rj If ^ x e M>o and Xi = 0, then a;q(')^ < V{x). 

Proof. For the proof we use the following notation: For a matrix M we denote by Mi,,j the 
matrix obtained by restricting M to its rows indexed by i, . . . ,j. Similarly, we denote by 
Mi,,j^k..e the matrix M restricted to the corresponding rows and columns. 

Let Q G denote the matrix whose columns are q*^^^^, . . . , q^")^. Note that IQ = 1. 

We have Q =~A*D where D G M"^" is the diagonal matrix with 1/Ai = (l^*)i- Further, 
we define the matrix P G {-1, 0, with 



P 



(-1 -1 -1 

1 

1 

V 



-1 -1\ 





1 / 



Stabilization of Branching Queueing Networks 



W.l.o.g. let Xi = 0. With the notation above and writing x := {x2, ■ ■ ■ ,Xn) G ^>o > ^he 
lemma states that 

there is no S e K"^^ with x ^0 and xQ2..nP < . (7) 
We use Farkas' lemma for the proof: 

► Lemma 13 (Farkas' lemma). Let M e M"^" and b E M". Then exactly one of the 
following is true: 

1. There exists x G M'" with xM < and xb^ > 0. 

2. There exists y G M" with My'^ = b^ and y >0. 

Using Farkas' lemma we prove (|7| by exhibiting 6 > and y > such that Q2..nPy^ — ■ 
We choose 

b^ := (A2..„,2..„)*1^>1^>0^. 

We have IQ — 1, which implies IQ^^ = IQQ^^ = 1, hence P{Q^^)2..n = Q^^ — E where 
E e {0,1}"^" denotes the matrix with 1 in the first row and in the other rows. It follows 

P{Q-%..nP^Q-^P- (8) 

Setting 

:= {Q-%..nPb^ 

we therefore have 

Q2..nPy'^ = Q2..nP{Q-%..nPb^ ^ Q2..nQ-^Pb^ = P2..nb^ - b^ , 

as desired. It remains to prove y > 0. As Q = A*D, we have Q^^ = D~^{I — A) with 
nonnegative, so it suffices to prove that (/ - A)2..nPb'^ > 0^. Indeed we have 

{I~A)2..nPb^ = {I-A2..nP)b^ = b^-A2..„P6^ > b^ - A2..n,2..nb^ = 1^, 

where the inequality holds as b^ and the first column of A2,,n are nonnegative and the first 
row of P is negative. 

Recall the following lemma: 

Lemma [8} There is 7 > such that we have 

A{x){V'{x))^ < forallx^O 
and all suhgradient vectors V'{x) ofV at x. More precisely, one can choose 
7 := min(/Xi - Aj)/ 

i 

where a^"^"^"^ is the ith column of A* . 

Proposition |4] follows from Lemma |8] using exactly the reasoning from theorem 1 and 
lemma 5 of [TU] . 
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A.3 Proof of Lemma [TT] 



Lemma 11 Assume that O is a memoryless ergodic scheduler. For every ^ G S there is a 



constant Of such that for almost all runs uj of JVq the limit 



iim — ^ 

t-^oo t 

exists and is equal to O^. There is S < 1 such that {S,0^ | € S) solves the traffic LP. 
Moreover, for every i g {1, . . . , n} the utilization pi of the queue i in A/e is equal to 



We divide our proof of Lemma 1 1 into four claims 



► Claim 1. For every ^ € S there is a constant such that for almost all runs uj of A/e the 
limit 

iim — 

t— foo t 

exists and is equal to O^. 

Proof. Given an action ^ S S and i > 1, denote by a random variable giving the number 
of times the action ^ is fired between the i-th and i + 1-st visit to (note that the network 
starts with all empty queues so the first visit happens at time 0). Observe that all are 
identically distributed. Also, as the expected return time to is finite and there are only 
finitely many distinct rates (i.e. the expected time between two state transitions is bounded 
from below by a positive constant), the expectation E is finite. By the strong law 

of large numbers (see e.g. [1]), there is a constant such that for almost all runs u! the 

limit lim„_j.oo ^'"'"^ ' ^ ^ exists and is equal to M^. For i > 1 we define Ri to be a random 
variable giving the total time between the i-th and i + 1-st visits to 0. Note that i?i is 
precisely the variable R giving the return time to 0. By the strong law of large numbers, for 

almost all runs uj the limit lim„^oo exists and is equal to E = E [i?] < oo. 

Now note that for t > and n > 1 satisfying J^^Zi Ri{oj) < t < X^ILi ^ii'^) (here for 
n = 1 we assume X]r=i ^ii^) — 0) we have 

The limit of the rightmost term of ^ exists: 



E[R] n^oo n Z^Li Z]"=l^»('^) 

Similarly, the limit of the left most term of ^ exists: 



E [i?] I n-i-oo n— 1 n->oo n I n-s-oo ^"^j^ i?j(a;) n-s-oo ^^^i Ri{uj) 



Thus by basic properties of limits, the limit limf_).oo — ^ — exists and is equal to 



E[R] ■ 
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The following two claims show that the frequencies indeed solve the traffic LP with 6 < 1. 
► Claim 2. For every j G {1, . . . ,n} we have 

Eo« = EEoc-^o- (10) 

Here we assume (in order to simplify notation) that the external source is a (symbolic) 
queue 0, with action set S = {t} where l has rate /Xq and a production function Probo (thus 
O, ■ A,j = cxj). 

Proof. Denote by O^Jj.^ (w) the number of jobs produced for queue j by firing the action 
The crucial observation is that for almost all runs oj we have 

t— >-CSO t t— >-CSO t 

Indeed, JZ^ei:, Of{oj) < X;Lo E^eE, OfXji^^) for all t and w because X;Lo Ecge^ ^fXji^) 
is precisely the number of jobs that enter the queue j up to time t. On the other hand, 
Er=o Efes- — E{es *^{'*('^) fo'^ infinitely many f and almost all uj because (almost 

surely) queue j becomes empty infinitely many times. 
We obtain that 

V = V lim ^ 

^ * ^ f-!-00 t 



t->(X) t 

j.^ ELoEceE, Qc"4M 

t— )-CSO i 

i=o ceEi 

EEl-^-Hm^ 
licfe*^°° * Of (a;) 



i=o ceEi 

Here the last equality follows from Claim 1 and the fact that lim^-^oo '■^J is the average 
number of jobs for queue j produced by firing i.e., Aqj. M 

► Claim 3. For every i G {1, . . . , n} we have 

E|eE/^ < 1 (11) 

Mi 

E E c>« 

Proof. Intuitively, we show that — h 7r(0) is equal to the proportion of time in which 

either the queue i operates a job or all queues are empty. As this proportion is at most 1 
and 7r(0) > 0, we obtain the desired result. 

Formally, denote by T„ the total amount of time in which the first n jobs are processed 
by queue i (by this we mean the total time from the beginning of the computation needed to 
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finish n jobs using the "machine" operating queue i). Denote by Nj the total time it takes to 
process the j-th job from queue i (by this we mean the time the "machine" actively spends 
by processing the job) . 

Observe that for almost all runs uj we have that lim„^oo — is the average 

processing time of the queue i which is , and that ^ "^^ is equal to 



<T„[u,) 



which means that by Claim 1, 



lim — — — 



lim 

n— ^oo 



Tniuj) 



Thus 



lim 

n— >-oo 



lim 

n— >oo 



lim — — — - 



(12) 



Denote by En the total amount of time the network spends in state before queue i finishes 
n jobs (by this we mean the sum of all time intervals up to time T„ in which all queues are 
simultaneously idle before n jobs are finished by the "machine" operating queue i). Intuitively, 
lim„_>.oo ^"1^1 is equal to the proportion of time spent in which is equal to 7r(0) > 0. More 
precisely, by the Ergodic Theorem, see Theorem 3.8.1 of [2S], for almost all runs lj we have 



< 7r(0) = 



1 /■* 

lim - / I[x{uj) = 0]{s)ds = lim 



Finally, by ([12| and ( |13| , 

Ese^^ < ^^^^-^^ +.(0) 



Mi 



Mi 



• lim 



lim 



(13) 



Eniio) ^. j:]=iNj{oj) + En{iu) 

— = 1™ — — 1 



► Claim 4. For every i e {1, . . . , n}, pi 



Proof. By the Ergodic Theorem, see Theorem 3.8.1 of PS^, and by (12 1, for almost all runs 
Lu we have 



= lim Pr(a;,(t) y^O) = V n{x) 

t^oo ^ — ^ 

= lim - / I[x,{uj) ^ 0]{s)ds -- 



lim 



This finishes a proof of Lemma |11[ 



